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1.0  BACKGROUND 

Reliability  of  launch  systems  is  a  current  and  historic  concern  in  the  space  launch  business. 
Program  Managers,  engineers  and  analysts  have  striven  to  increase  launch  reliability.  As 
far  back  as  1960,  General  Power,  Commander  in  Chief  of  the  Air  Force  Strategic  Air 
Command,  warned  that  reliability  improvements  depended  on  an  effective  test  program. 
Although  engineers  continuously  sought  product  improvements  to  increase  reliability,  the 
Intercontinental  Ballistic  Missile  Program  suffered  from  failure  after  failure.  Headquarters 
Air  Force  agreed  with  General  Powers  and  encouraged  more  testing.  However,  they 
warned  that  the  missiles  were  being  “over  exercised”  suggesting  the  extensive  testing  may 
be  negatively  impacting  reliability.  The  unreliability  of  the  Atlas  caused  General  LeMay, 
Air  Force  Chief  of  Staff,  to  form  a  board  to  investigate  the  causes  of  the  continued 
failures.  The  board  found  many  factors  for  the  failures: 

3  inadequate  testing, 

3  poor  facilities, 

3  insufficient  training, 

3  insufficient  technical  data, 

3  poor  configuration  control, 

3  poor  quality  control.1 

In  response  to  this  study,  the  Air  Force  formed  the  Golden  Ram  Program.  This  program 
instituted  all  the  findings.  Prior  to  the  study  only  3  of  16  Atlas  were  successful. 
Afterwards  23  of  25  were  successful.  The  Air  Force  has  not  applied  this  same  type  of 
rigorous  study  to  the  space  launch  fleet,  although  the  same  factors  apply. 

The  Air  Force  faces  a  tough  future  in  space  launch  systems.  Launch  rates  are  decreasing 
as  budgets  decrease  and  existing  satellite  systems  out-live  their  planned  lifetime. 
Additionally,  Air  Force  satellites  are  not  all  common  bus  systems.  In  some  cases,  the 
systems  are  one  of  a  kind.  This  makes  the  price  of  failure  extremely  high.  Even  in  satellite 
systems  that  may  have  a  manufacturing  rate  of  two  per  year,  a  failure  may  mean  six 
months  or  longer  before  a  replacement  is  available.  This  down-time  hurts  the 
effectiveness  of  space  systems  in  operational  mission  support  and  potentially  compromises 
the  ability  of  our  space  systems  to  support  mission  readiness  in  future  years. 

The  purpose  of  this  study  is  to  consider  these  and  other  factors,  or  metrics,  and 
recommend  how  they  may  contribute  to  the  overall  reliability  assessment  of  space  launch 
systems.  This  study  assesses  how  a  set  of  metrics  can  form  the  basis  for  reliability 
assessment  of  a  typical  space  launch  system. 


'Ballistic  Missiles,  Jacob  Neufeld.  Office  of  Air  Force  History,  US  Air  Force,  1990,  p  217. 
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2.0  RELIABILITY  UNCERTAINTY 

Analysts  have  developed  complex  and  detailed  methods  for  calculating  the  reliability  of 
launch  systems.  However  these  methods  have  lacked  the  necessary  data  for  a  statistically 
significant  calculation.  The  universal  set  is  too  small  to  derive  any  conclusions  without 
significant  caveats. 

Without  a  significantly  large  data  set,  engineers  have  designed  conservatism  into  their 
systems.  Due  to  a  lack  of  significant  data,  engineers  “over  design”  to  improve  reliability. 
This  is  a  contributing  factor  to  high  launch  costs.  Additionally,  as  new  generation  launch 
vehicles  evolve  from  previous  ones,  the  “over  design”  slowly  erodes.  Much  of  this 
erosion  is  justifiable  since  the  data  set  is  increasing.  However,  the  universal  set  is  still 
much  too  small  for  accurate  assessments. 

In  recent  years,  the  federal  government  encouraged  contractors  to  develop  launch  systems 
using  commercial  practices.  The  rationale  was  for  the  manufacturers  to  become 
commercially  competitive,  reduce  costs  for  government  launches,  and  increase  the  number 
of  launches.  Although  this  has,  in  some  cases,  proven  true  (for  example.  Medium  Launch 
Vehicle  -  III),  there  are  significant  cases  where  this  approach  has  not  helped  to  reduce 
costs  or  increase  launch  rate.  Even  in  cases  where  the  launch  rate  has  increased,  it  has  not 
increased  in  a  statistically  significant  amount.  The  final  results  are  launch  systems  no  more 
reliable  than  previous  ones.  Additionally,  by  encouraging  competitive  practices,  the  Air 
Force  owns  less  data  and  has  a  lesser  ability  to  predict  the  success  of  future  missions. 

Because  of  the  low  use  rate,  space  launch  systems  are  a  special  case.  From  a  reliability 
standpoint,  factors  that  analysts  and  engineers  normally  dismiss  as  irrelevant  require 
consideration.  Since  the  high-level  data  set  is  not  statistically  significant,  analyses  must 
include  the  third,  fourth,  and  fifth  level  factors.  Every  factor  requires  consideration  unless 
substantial  proof  shows  no  effect  on  the  overall  launch  system  reliability.  Unfortunately, 
the  Air  Force  has  never  analyzed  these  lower  level  metrics  and  how  they  apply  to  space 
launches. 
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3.0  COMPONENTS  OF  LAUNCH  RELIABILITY 

Although  the  vehicular  reliability  of  the  launch  system  (Rv)  is  very  important,  the  launch 
support  reliability  (Rs)  is,  in  some  cases,  even  more  important.  The  actual  reliability  (Ra) 
of  the  complete  launch  system  is  a  function  of  both  Rv  and  Rs. 

3.1  Vehicular  Reliability  (Rv) 

Launch  vehicle  manufacturers  advertise  their  Rv.  This  is  a  selling  point  for  their  systems. 
However,  different  contractors  measure  Rv  in  different  ways.  For  example,  Rv  may  be  a 
quantity  in  their  contract.  Perhaps  their  current  Rv  is  for  a  few  vehicles  (e.g.,  the  last  ten 
launches  since  incorporating  certain  product  improvements).  These  values  are  purely  right 
and  proper,  although  one  must  realize  what  they  reflect.  Generally,  reliability  numbers 
listed  in  various  payload  users'  manuals  are  a  combination  of  system  component 
reliabilities. 

The  contractors  make  vast  assumptions  in  their  reliability  calculations  that  this  paper 
contends  are  not  true.  For  example,  they  assume  the  reliability  of  the  launch  crew  is  1.0. 
Logically,  they  would  portend  that  computer  systems  are  sophisticated  enough  to  scrub  a 
launch  if  the  launch  crew  makes  an  error  in  judgment.  This  is  not  so  for  the  previous 
Titan  IV  failure  at  Vandenberg.2 3 4  Although  authorities  did  not  hold  the  launch  crew  solely 
responsible  for  the  failure,  there  were  several  decision  points  throughout  the  launch  cycle 
where  any  engineer  could  have  exercised  judgment  and  stopped  the  processing.  Should 
the  launch  crew  have  seen  the  failure  and  stopped  the  mission?  No  one  knows  for  sure. 
The  Office  of  Technology  Assessment  staff  received  a  briefing  at  Vandenberg  following  a 
similar  Titan  34D  failure  several  years  earlier.  Vandenberg  launch  crews  told  them  that 
even  with  additional  testing  instituted  after  the  Titan  34D  failure,  engineers  and  operators 
at  the  launch  site  were  not  sure  they  could  have  prevented  the  failure.  They  had  never 
faced  that  exact  combination  of  factors.  They  may  never  face  that  combination  again. 
This  was  proven  true  by  the  similar  Titan  IV  failure.  Although  the  failure  mechanism  was 
similar,  the  details  were  so  different  from  the  Titan  34D  incident  that  the  same  did  not 
apply.  Because  of  this  environment  of  uncertainty,  there  is  no  rational  way  an  analyst  can 
assume  the  reliability  for  the  launch  crew  to  be  1 ,0.34 

Let's  consider  another  example.  The  US  Ranges  are  some  of  the  best  in  the  world.  Once 
the  current  upgrade  is  complete,  they  will  have  the  most  reliable  equipment;  most 
experienced  personnel;  and  most  advanced  architecture  available.  However,  is  their 
reliability  1.0?  They  would  enjoy  such  a  reputation.  Even  the  most  ardent  Range  fan  will 
admit  their  systems  have  bad  days.  However,  the  reliability  values  in  the  Payload  Users 
Manuals  assume  the  Range  reliability  is  1.0. 


2  The  Titan  IV  failure  was  attributed  to  a  solid  rocket  motor  debonding.  This  was  very  similar  to  the  Titan  34D  failure  in  1986.  Although 
testing  and  screening  for  these  debonds  had  been  significantly  increased  since  the  1986  failure,  the  ability  to  identify  future  failure  was 
obviously  not  100%. 

3Reducing  Launch  Operations  Costs,  New  Technologies  and  Practices,  Congress  of  the  United  States,  Office  of  Technology  Assessment 
September  1988 

4  For  a  detailed  discussion  of  this  Titan  34D  failure  and  a  comparison  to  the  Challenger  failure,  see  Guardians,  Curt  Pebbles,  1987. 
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Examples  of  this  over  simplification  are  numerous.  All  outside  influences  contribute  to  the 
reliability  determination  for  a  given  launch  vehicle  system.  Air  Force  Space  Command  has 
repeatedly  stated  that  the  real  concern  of  the  program  manager  or  operator  is:  "What  is 
the  reliability  of  this  system  placing  my  satellite  in  the  orbit  where  I  need  it?"5  This 
reliability  is  not  what  is  in  the  Payload  Users  Guides.  To  determine  this  reliability,  we 
must  consider  Support  Reliability  (Rs). 

3.2  Support  Reliability  (Rs) 

The  mission  of  the  launch  vehicle  from  the  satellite  program  manager’s  point  of  view  is  to 
deliver  a  satellite  to  a  defined  orbit.  Regardless  of  the  sum  of  the  piece  part  Rv  values, 
regardless  of  the  advertised  reliability,  this  final  placement  of  the  satellite  is  the  true 
purpose  of  the  launch  system.  Much  to  the  chagrin  of  the  launch  vehicle  manufacturers, 
this  berates  the  launch  vehicle  to  nothing  more  than  a  transport.  The  launch  vehicle  is  a 
single  component  in  a  complex  system  required  to  place  the  satellite  in  orbit.6 

By  considering  the  components  of  this  complex  system,  we  determine  the  factors  of  Rs. 

1.  Facilities:  The  facilities  that  support  launch  are  critical.  Although  the 
computer  system  can  shut  down  a  launch  in  the  final  seconds  if  there  is  a 
facility  problem,  the  impact  of  such  a  shutdown  is  significant  and  not 
entirely  understood.  The  Titan  34D  launch  that  crews  scrubbed  within 
seconds  of  liftoff  due  to  a  water  valve  not  being  replaced  is  a  good  example 
of  the  importance  of  facilities.  In  that  case,  the  prevalves  had  opened  on 
the  Aerojet  engines  requiring  a  launch  within  72  hours.  Under  time-critical 
conditions  like  this,  reliability  calculations  become  extremely  difficult.  Any 
accurate  reliability  figure  must  consider  other  factors  besides  Rv. 

2.  Contractor  Crew:  Much  of  the  corporate  memory  is  in  the  minds  of  the 
launch  vehicle  processing  contractors.  Although  the  contractors  and  the 
Air  Force  document  decisions  very  carefully,  nobody  documents  the 
engineering  judgments  that  led  to  those  decisions.  As  the  contractor  crews 
retire,  shrink  in  size,  or  change  jobs,  the  reliability  does  not  go  unaffected. 
An  Rs  value  is  necessary. 

3.  Air  Force  Crew:  One  of  the  main  objectives  of  the  Air  Force  Space 
Launch  Squadrons  is  to  provide  mission  assurance  and  safety  oversight  of 
the  launch  processing.  Since  very  little  of  the  processing  is  taught  to  the 
Air  Force  crews,  and  no  automated  database  of  histories  (such  as  EMDAS 
and  CAMS)  exist.  There  is  no  educational  process  besides  OJT  that  allows 
the  Air  Force  crews  to  make  top  quality  assessments  of  the  contractor’s 
work.  They  are,  more  often  than  not,  in  a  "trust  me"  mode  with  the  launch 
processing  contractors.  Because  of  this,  one  could  argue  that  the  Space 


5  Various  phone  conversations  with  Headquarters  Air  Force  Space  Command  Offices,  April  -  May  1994. 

6There  have  been  some  instances  where  the  launch  vehicle  was  as  much  a  part  of  the  mission  as  the  payload  it  earned.  Those  experiments, 
however,  are  few. 
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Launch  Squadron  provides  no  value  added.  This  is  emphatically  not  the 
case.s  The  Space  Launch  Squadron  provides  the  satellite  program  manager 
and  the  operator  with  a  presence  that  ensures  contractors  follow  proper 
procedures.  This  becomes  even  more  important  as  launch  rates  decrease 
and  satellites  are  unique.  The  Air  Force  Crew  is  a  critical  part  of  the  Rs 
equation  when  the  entire  system  is  taken  into  account. 

4.  Range:  As  mentioned  earlier,  the  Range  resources  are  critical  to  the 
success  of  any  launch.  These  systems  must  be  in  top  condition. 
Additionally,  the  Range  crews  must  be  trained  on  the  launch  profile  and 
any  peculiar  details  of  the  launch.  Given  the  case  of  the  Titan  34D  scrub 
above,  the  Range  capability  to  reorganize,  reconfigure,  and  be  ready  to 
perform  in  a  very  limited  time  is  essential.  This  process  may  warrants 
considerable  attention  since  it  is  not  routine. 

5.  Weather:  This  is  a  very  uncertain  parameter  contributing  to  the 
probability  of  a  launch  success.  As  it  applies  to  reliability,  one  needs  to 
understand  how  accurately  the  weather  is  predicted.  If  there  is  a  plane 
flying  in  the  region  reporting  the  weather  conditions,  the  reliability  of  the 
reports  may  be  higher  than  if  the  reports  are  written  based  on  balloon  data. 
The  aircraft  report  reliability  may  increase  if  a  weather  expert  is  on  the 
aircraft. 


These  are  the  primary  factors  of  Rs.  As  Rs  factors  are  tracked,  recorded,  and  used,  other 
factors  may  become  apparent.  This  set  is  a  good  starting  set.  Together  with  Rv,  we  can 
determine  the  actual  reliability  of  the  system. 

3.3  Actual  Reliability  (Ra) 

The  actual  reliability  of  a  launch  system  is  what  the  satellite  program  manager  wants  to 
know.  This  value  is  a  combination  of  the  Rv  and  Rs.  An  understanding  of  this 
relationship  is  important.  A  linear  relationship  Ra  =  (Rv  +  Rs)/2  is  an  easy  way  to 
envision  the  use  of  Rv  and  Rs.  The  actual  relationship  between  the  two  factors  is  beyond 
the  scope  of  this  study  and  would  require  further  data  gathering  and  manipulation. 

The  program  manager  can  follow  Ra  throughout  the  development  of  the  launch  system 
until  launch.  Air  Force  Launch  Crews  could  perform  continuous  assessment  of  Ra  until 
the  final  launch  commit  decision.  Although  the  Rv  value  is  fairly  rigid  once  the  launch 
vehicle  is  at  the  launch  site,  it  may  change  as  components  are  removed  and  replaced.  On 
the  other  hand,  Rs  may  change  drastically  from  day  to  day  throughout  the  launch  cycle. 
Given  the  proper  ability  to  monitor  and  track  Rs,  the  program  manager  can  have  the 
highest  probability  of  success  for  any  given  launch. 


4.0  TRACKING  AND  MONITORING  Rs 

To  determine  the  data  needed  for  each  of  the  factors  of  Rs,  a  simple  analysis  of  each 
factor  is  necessary.  Each  factor  has  several  parts  that  effect  launch.  The  parts  effect 
launch  availability,  timeliness,  responsiveness,  or  certainty.  Other  studies  have  identified 
these  as  critical  items.7  First  consider  definitions  of  these  terms: 


Launch  Availability  ' 

This  is  a  measure  of  the  impact  on  the  ability  of  the  system  to  launch.  For  example,  does  a 
failure  of  this  item  cause  the  system  not  to  launch?  A  practical  example  of  this  would  be 
the  launch  pad  crane.  Certainly  without  it,  there  is  no  launch  capability.  Therefore  a 
failure  in  the  crane  would  cause  concern  and  would  impact  the  Rs  of  the  system. 


Launch  Timeliness 

This  is  a  measure  of  the  impact  on  the  ability  of  the  system  to  meet  the  required  launch 
window.  For  example,  does  a  failure  of  the  item  cause  the  system  not  to  launch  within  the 
expected  window?  A  practical  example  of  this  would  be  a  range  radar.  If  the  radar  goes 
down,  it  may  take  too  long  to  repair  causing  a  missed  launch  window. 


Launch  Responsiveness 

This  is  a  measure  of  the  impact  on  the  ability  to  respond  to  a  launch  on  need  or  call-up 
requirement.  For  example,  if  a  program  has  a  launch  on  need  requirement,  does  a  failure 
of  this  item  affect  the  system’s  ability  to  meet  that  requirement.  If  a  program  has  a  call-up 
requirement,  the  same  criteria  hold.  A  practical  example  of  this  would  be  a  transporter.  If 
a  program  had  only  one  transporter  and  it  failed,  there  would  be  doubts  about  the  ability 
to  meet  a  launch  on  need  requirement. 


Launch  Certainty 

This  is  a  measure  of  the  impact  on  the  ability  of  the  system  to  launch  when  expected. 
How  certain  is  the  program  manager  that  the  launch  window  will  be  met?  For  example, 
the  weather  plays  a  vital  role  in  this  factor.  If  the  predicted  weather  is  marginal,  and  the 
ability  to  predict  it  is  not  perfect,  then  this  factor  may  be  very  large. 

One  can  see  how  a  given  support  item  may  fit  into  more  than  one  of  the  criteria  above. 
The  key  is  that  if  it  fits  into  any  one,  then  it  provides  some  kind  of  impact  on  Rs  and  needs 
to  be  monitored. 

4.1  Parameters  Within  Each  Factor 

Each  of  the  factors  of  Rs  identified  in  Section  3.2  have  parameters  that  need  to  be 
monitored.  These  parameters  must  fit  the  above  criteria  in  some  fashion.  Initially,  a 


^  These  items  and  others  are  described  in  detail  in  Military  Space  Forces,  John  M.  Collins,  1989. 
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parameter  may  be  considered  critical.  If,  after  several  launches  of  monitoring  it,  the  crews 
determine  it  has  no  impact  on  any  of  the  criteria,  it  no  longer  necessitates  monitoring.  On 
the  other  hand,  as  certain  parameters  are  monitored,  crews  may  discover  other  parameters 
that  become  critical.  These  would  be  added  for  future  launches.  Based  on  the  initial 
analysis,  the  following  paragraphs  outline  the  initial  set  of  parameters  necessary  to  develop 
an  Rs  value. 


Facilities 

There  are  three  parameters  within  facilities  that  affect  Rs.  The  number  of  supporting 
facilities  is  important.  Obviously,  the  higher  the  number,  the  higher  the  probability  of 
affecting  the  criteria.  The  amount  of  time  since  the  last  maintenance  efforts  in  each  facility 
must  be  monitored.  The  criticality  of  clean  rooms  is  an  important  factor  since  there  are 
many  ways  the  lack  of  a  clean  room  could  impact  the  criteria. 


FACILITIES  PARAMETERS 
Number  of  supporting  facilities 
Time  since  last  maintenance  efforts 
Criticality  of  Clean  Rooms 


Contractor  Crew 

The  contractor  crew  have  several  items  which  impact  their  reliability.  The  number  of  new 
people  and  the  number  of  losses  since  the  last  launch  is  critical.  Amount  of  training 
performed  by  each  crew  member  in  specific  skills  is  important.  Other  contributing  factors 
include  the  length  of  the  shifts  and  the  length  of  the  launch  cycle. 
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CONTRACTOR  CREW  PARAMETERS 
Number  of  new  personnel  since  last  launch 
Number  of  losses  since  last  launch 
Training  in  specific  skills  areas 
Length  of  shifts  worked 
Length  of  launch  cycle 


Air  Force  Crew 

The  Air  Force  Crew  is  similar  to  the  contractor  crew.  They  have  basically  the  same 
parameters  for  monitoring.  However,  the  criteria  are  weighed  differently.  This  is 
predominantly  driven  by  the  frequency  of  turnover  in  Air  Force  personnel.  Where 
contractor  crews  tend  to  stay  with  the  launch  team  for  a  significantly  long  time  (20  years), 
the  Air  Force  crew  is  short  lived  (3-5  years).  This  is  reflected  in  their  duties;  Air  Force 
crews  provide  oversight  whereas  the  contractor  crews  perform  the  actual  work. 
Additionally,  because  of  the  high  turnover  rate,  training  is  very  important. 


AIR  FORCE  CREW  PARAMETERS 
Number  of  new  personnel  since  last  launch 
Number  of  losses  since  last  launch 
Training  in  specific  skills  areas 
Length  of  shifts  worked 
Length  of  launch  cycle 


Range 

The  value  and  potential  impact  of  the  Range  is  too  extensive  to  encapsulate  in  just  a  few 
short  lines.  Each  program  needs  to  assess  which  items  are  significant  and  to  what  degree. 
A  start  would  be  to  monitor  all  the  mandatory  items  listed  in  the  Program  Requirements 
Document.  In  addition  to  these  items,  there  are  other  concerns  a  program  may  want  to 
track.  These  would  include  similar  crew  concerns  as  the  contractor  and  Air  Force  items 


listed  above.  Length  of  time  since  the  last  maintenance  performed  and  redundancy  on 
critical  equipment  is  important. 


RANGE  PARAMETERS 

Number  of  new  personnel  since  last  launch 

Number  of  losses  since  last  launch 

Amount  of  training  on  mission 

Length  of  shifts  worked 

Amount  of  mandatory  resources  required 

Time  since  last  maintenance  on  madatory  items 

Amount  of  redundancy  in  mandatory  items 


Weather 

Although  some  may  consider  this  a  part  of  the  Range  resources,  it  has  been  significantly 
critical  in  the  past  to  warrant  separate  consideration.  Data  has  shown  that  Weather  has 
been  responsible  for  as  much  as  11%  of  the  launch  delays  recorded.  This  number  may  be 
higher  except  that  some  delays  were  shadowed  by  additional  problems  with  Range  or  the 
launch  system.8  Weather  parameters  must  be  well  defined.  They  include  the  number  of 
radar  used,  the  number  of  aircraft  used,  the  susceptibility  of  the  system  to  weather,  the 
confidence  of  the  forecast,  and  the  amount  of  training  the  weather  personnel  have  received 
on  the  program.  For  example,  if  weather  balloons  are  used  for  measurements  of  winds 
aloft,  then  the  reliability  of  that  forecast  would  be  less  than  if  radar  and  aircraft  are  used  to 
provide  measurements.  Additionally,  the  older  the  forecast,  the  lesser  the  reliability. 


8  NSIA  Launch  Responsiveness  Launch  System  Panel  Results,  June  1994 


WEATHER  PARAMETERS 
Time  since  last  forecast 
Sophistication  of  measurement  equipment 
Training  of  Weather  Crews 
Susceptability  of  mission  to  weather 


4.2  Application  of  the  Parameters 

Each  program  evaluates  parameters  differently,  so  any  example  we  consider  is 
hypothetical.  For  this  study,  assume  a  program  launches  from  only  one  launch  site. 

Each  launch  system  has  a  Readiness  Review  Cycle  it  passes  through  as  the  system  is 
prepared  for  launch.  Each  of  the  Readiness  Reviews  are  linked  to  a  milestone  event  in  the 
launch  preparation  activities.  At  each  one  of  these  Readiness  Reviews,  an  Rs  value  should 
be  evaluated  and  assessed.  As  the  launch  approaches,  there  should  be  a  continual,  perhaps 
hourly,  updating  of  Rs  until  the  final  commit  to  launch. 

A  typical  launch  cycle  may  have  the  following  reviews: 9 


Postproduction  Review 

This  review  is  conducted  at  the  contractor’s  manufacturing  facility  at  the  end  of 
production.  The  primary  purpose  of  this  review  is  to  assure  the  hardware  is  ready 
for  shipment  to  the  launch  site. 


Pre-  Vehicle-On-Stand  Review 

This  review  is  held  after  the  preliminary  receipt  and  checkout  of  the  launch  system 
is  complete.  The  primary  purpose  of  the  review  is  to  assure  the  system  is  ready  to 
be  placed  on  the  launch  pad. 


Launch  Site  Readiness  Review 

This  review  is  held  prior  to  the  erection  and  mate  of  the  upper  stage  and  satellite. 
The  primary  purpose  of  the  review  is  to  assess  progress  since  the  Pre-Vehicle-On- 


9  This  list  of  reviews  reflects  those  described  in  the  Commercial  Delta  II  Payload  Planners  Guide,  McDonnell  Douglas  Commercial  Delta, 
Inc.,  December  1989. 
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Stand  Review.  This  review  verifies  readiness  of  all  system  components  for  transfer 
of  the  satellite  to  the  launch  pad. 


Flight  Readiness  Review 

This  review  is  held  upon  completion  of  combined  system  testing.  It  verifies  the 
satellite  and  launch  system  are  ready  for  countdown  and  launch.  The  purpose  of 
this  review  is  to  get  corporate  approval  from  all  the  various  agencies  supporting 
the  launch  that  they  are  ready  to  proceed  into  final  launch  preparations. 


Launch  Readiness  Review 

This  review  is  held  the  day  prior  to  beginning  the  final  launch  countdown.  This 
forum  provides  all  agencies  an  opportunity  to  discuss  any  concerns  regarding 
launch.  It  also  assures  all  agencies  are  prepared  to  proceed.  The  purpose  of  this 
review  is  to  get  approval  to  start  the  terminal  countdown. 

Various  factors  of  Rs  carry  different  importance  at  different  reviews.  For  example. 
Weather  parameters  do  not  have  much  bearing  on  the  readiness  for  the  system  to  be  sent 
to  the  launch  site.  However,  the  readiness  is  affected  by  contractor  crew  training.  The 
following  chart  is  a  sample  of  how  the  parameters  could  be  tracked  for  each  review. 


This  type  of  matrix  would  provide  the  mission  director  the  ability  to  monitor  the  progress 
of  each  Reliability  area.  Each  area  could  be  tracked  throughout  the  launch  processing 
sequence.  This  would  identify  risks  in  areas  that  are  not  currently  monitored  in  this 
manner.  It  is  important  to  note  these  items  are  currently  being  tracked  in  some  fashion. 
For  example,  the  readiness  of  the  Range  is  constantly  tracked  and  assessed.  The 
uniqueness  of  this  approach  is  that  it  assesses  the  readiness  of  the  Range  to  support  one 


particular  mission.  Although  the  Range  tracks  and  evaluates  the  readiness  of  each 
mandatory  item,  the  reliability  of  a  system  is  directly  related  to  how  many  mandatory  items 
are  necessary  for  that  system  to  launch. 

This  same  approach  would  hold  true  for  Weather.  Although  Weather  crews  have  specific 
guidelines  that  tell  them  what  weather  situations  are  most  important  for  a  given  mission, 
how  susceptible  that  mission  is  to  those  weather  situations  is  an  evaluation  only  performed 
by  the  Mission  Director.  Obviously,  it  will  rely  on  how  well  the  Weather  Crews  are 
trained  to  respond  to  the  peculiarities  of  the  mission. 

This  approach  puts  the  responsibility  of  the  success  of  the  mission  where  it  should  be— 
with  the  Mission  Director.  It  allows  the  crews  a  method  of  identifying  areas  where  they 
perceive  problems  exist.  If  they  do  not  have  enough  training  to  perform  the  mission,  then 
the  training  is  rated  low.  If  they  do  not  have  the  personnel  they  need  or  if  they  are 
working  longer  periods  than  what  is  healthy,  then  those  ratings  are  low. 

4.3  EVALUATION  TECHNIQUES 

Using  the  parameters  in  the  hypothetical  program,  each  needs  to  be  assessed  against  the 
criteria  of  launch  availability,  timeliness,  responsiveness,  and  certainty.  This  should  be  an 
effort  performed  by  the  Space  Launch  Squadrons.  This  assessment  would  provide  a  scale 
for  the  detailed  reliability  analysis.  The  scale  shown  here,  as  with  the  parameters,  is  only  a 
hypothetical  approach.  Note  that  the  effect  of  some  of  the  parameters  on  some  of  the 
criteria  can  not  change  without  a  significant  redesign  of  the  program.  Others  can  change 
simply  with  quality  planning.  We  will  show  later  how  the  evaluation  will  change 
throughout  the  launch  process. 


4.3.1  FACILITIES  EXAMPLE 

Number  of  Supporting  Facilities 

Rating  Description 

0  Has  no  facility  requirements. 

1  Only  requires  single  facility  or  has  no  special  facility  requirements. 

2  Requires  a  few  facilities,  but  has  flexibility  in  where  those  facilities  exist. 

3  Requires  several  facilities.  Has  specific  requirements  that  are  unique,  but 
manageable  with  other  programs. 

4  Requires  several  facilities  that  are  well  defined.  Requires  modification  of 
existing  facilities  to  perform  successful  mission 

5  Requires  several  facilities  to  function  in  coordination  with  each  other.  Each 
facility  is  uniquely  tailored  to  the  specific  mission.  No  flexibility  if  a  facility  is 
not  available. 


I 

i 

Time  Since  Last  Maintenance  Efforts 
1  Rating  Description 

I  0  No  maintenance  scheduled  throughout  the  launch  cycle. 

1  All  maintenance  items  complete  within  the  previous  three  months. 

12  All  maintenance  items  complete  within  the  previous  six  months. 

3  All  maintenance  items  complete  within  the  previous  year.  80%  within  the 

previous  six  months. 

|  4  All  maintenance  items  complete  within  the  previous  year.  50%  within  the 

*  previous  six  months. 

5  Some  maintenance  performed  over  a  year  ago. 


1 
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Criticality  of  Clean  Rooms 
Rating  Description 

0  No  clean  room  required  for  mission  success. 

1  Mission  only  requires  special  handling  for  environmental  protection 

2  Mission  only  requires  an  environmental  protection  room. 

3  Mission  requires  a  visibly  clean  work  environment. 

4  Mission  requires  a  specific  level  clean  room  for  part  of  the  processing  period. 

5  Mission  requires  a  specific  level  clean  room  continuously  throughout  the  launch 

processing. 


4.3.2  EVALUATION  THROUGH  THE  LAUNCH  PROCESS 

Now  consider  how  the  above  criteria  might  be  evaluated  through  a  typical  launch  cycle. 

The  following  figure  shows  a  sample  evaluation  prior  to  the  postproduction  review. 


SAMPLE  MISSION 

POST 

PRODUCTION 

REVIEW 

PRE- 

VEHICLE- 

ON-STAND 

REVIEW 

LAUNCH  SITE 
READINESS 
REVIEW 

FLIGHT 

READINESS 

REVIEW 

LAUNCH 

READINESS 

REVIEW 

FACILITIES 

lailll 

NUMBER  OF  SUPPORTING 
FACILITIES 

4 

TIME  SINCE  LAST 
MAINTENANCE  EFFORTS 

4 

CRITICALITY  OF  CLEAN 
ROOMS 

3 

This  sample  shows  that  the  program  has  several  facilities  that  are  well  defined,  but  require 
modifications.  All  maintenance  has  been  completed  within  the  last  year.  The  mission 
requires  only  a  visibly  clean  work  environment  to  get  to  the  next  milestone.  The  facilities 
support  reliability  could  be  calculated  using  the  following  formula: 


15 


Rs{ facilities )  =  1  - 


=  27 


4  +  4  +  3 


A  more  thorough  analysis  will  be  necessary  to  determine  the  exact  relationship  of  each  of 
these  parameters.  Certain  programs  may  want  each  parameter  weighed  differently  or 
calculated  in  a  combination  of  parallel  and  serial  relationships  (see  Section  4.4).  However, 
for  this  example  a  simple  corrected  mean  calculation  is  sufficient  to  show  the 
methodology.  For  a  highly  reliable  system,  the  number  approaches  “1”.  The  method 
reveals  areas  of  vulnerability  allowing  directors  to  correct  situations  as  they  approach 
launch.  Additionally,  this  approach  will  identify  continuous  problem  areas  associated  with 
all  launches. 


As  the  program  enters  the  next  phase  they  hold  the  Pre-Vehicle-on-Stand  Review.  During 
this  review  the  reliability  is  once  again  considered.  As  before,  the  example  here  only 
considers  facilities.  However,  in  practice  all  areas  will  be  reconsidered  prior  to  each 
milestone  review. 


SAMPLE  MISSION 

POST 

PRODUCTION 

REVIEW 

PRE- 

VEHICLE- 

ON-STAND 

REVIEW 

LAUNCH  SRE 
READINESS 
REVIEW 

FLIGHT 

READINESS 

REVIEW 

LAUNCH 

READINESS 

REVIEW 

FACILITIES 

NUMBER  OF  SUPPORTING 
FACILITIES 

4 

3 

TIME  SINCE  LAST 
MAINTENANCE  EFFORTS 

4 

3 

CRITICALITY  OF  CLEAN 
ROOMS 

3 

1 

In  this  example,  we  see  changes  at  this  milestone.  Although  the  program  requires  several 
facilities,  they  are  more  well-defined  for  this  stage.  No  modifications  are  necessary.  The 
maintenance  crews  have  caught  up  on  more  of  the  maintenance  items,  although  this  is  still 
a  problem  area.  Only  environmental  covers  on  sensitive  surfaces  are  necessary  for  the 
next  phase,  so  this  area  is  rated  at  “1”.  This  yields  a  Facilities  Support  Reliability  of: 


Rs(facilities)  =  1  - 


3  +  3  +  1 


n 


=  53 


This  shows  that  the  reliability  related  to  the  facility  support  has  doubled  since  the  last 
review.  This  shows  a  good  trend  as  we  approach  launch.  The  next  milestone,  the  Launch 
Site  Readiness  Review,  is  usually  done  prior  to  bringing  the  satellite  out  to  the  launch  pad. 
Launch  controllers  are  usually  very  particular  about  making  sure  the  chances  of  a  launch  in 
the  near  future  are  very  high.  Therefore,  the  support  reliability  number  at  that  stage  is 
very  important.  Since  most  satellites  require  some  sort  of  clean  room  environment  on  the 
launch  pad,  the  Criticality  of  the  Clean  Room  becomes  significant.  Things  to  considered 


How  quickly  can  the  satellite  be  “safed”  if  the  power  is  lost  to  the  clean  room? 
How  sensitive  is  the  satellite  to  fluctuations  caused  by  the  extreme  temperatures? 
What  is  the  impact  if  we  lose  the  clean  room  facility  completely?  Do  we  not 
launch? 

These  questions  become  very  critical  at  this  stage.  The  answers  could  vary  significantly. 
In  our  example,  we  assume  the  satellite  requires  a  minimum  clean  room  facility  only 
during  certain  times  in  the  prelaunch  activities.  By  this  time  all  the  old  maintenance  items 
are  complete,  and  the  number  of  facilities  supporting  the  activities  has  dropped 
significantly.  This  gives  the  following  chart: 


SAMPLE  MISSION 


FACILITIES 

NUMBER  OF  SUPPORTING 

FACILITIES _ 

TIME  SINCE  LAST 
MAINTENANCE  EFFORTS 
CRITICALITY  OF  CLEAN 
ROOMS 


POST  PRE-  LAUNCH  SITE  FLIGHT  LAUNCH 

PRODUCTION  VEHICLE-  READINESS  READINESS  READINESS 

REVIEW  ON-STAND  REVIEW  REVIEW  REVIEW 


REVIEW 


Based  on  this  information  we  can  see  that  the  facility  support  reliability  changed  again. 


Rs(facilities )  =  1  - 


^2  +  2  +  4 
v  5 


/  3 


=.47 


As  we  approach  the  Flight  Readiness  Review,  the  clean  room  facility  is  no  longer 
important  since  the  payload  fairing  is  in  place.  This  milestone  is  usually  to  assess  the 
readiness  of  the  system  to  load  propellant  on  the  booster.  The  process  still  requires 
protection  from  the  elements.  By  this  time  all  maintenance  items  should  be  completed. 


By  this  time  we  are  approaching  a  critical  point  in  the  reliability  calculation.  Any 
extensive  effort  to  improve  the  reliability  numbers  from  this  point  on  will  likely  cause  a 
launch  delay.  The  calculated  Facility  Support  Reliability  at  the  Flight  Readiness  Review 
is: 


Rs(  facilities )  =  1  - 


2  +  1  +  2 


At  the  Launch  Readiness  Review,  the  number  of  facilities  supporting  the  operation  goes 
up  significantly,  however,  they  are  well  defined.  Because  of  the  large  volume,  the  number 
still  remains  high.  No  clean  room  is  necessary  for  any  further  operation,  and  all 
maintenance  items  are  complete. 


SAMPLE  MISSION 

POST 

PRODUCTION 

REVIEW 

PRE- 

VEHICLE- 

ON-STAND 

REVIEW 

LAUNCH  SITE 
READINESS 
REVIEW 

FLIGHT 

READINESS 

REVIEW 

LAUNCH 

READINESS 

REVIEW 

FACILITIES 

NUMBER  OF  SUPPORTING 
FACILITIES 

4 

3 

2 

2 

3 

TIME  SINCE  LAST 
MAINTENANCE  EFFORTS 

4 

3 

2 

1 

1 

CRITICALITY  OF  CLEAN 
ROOMS 

3 

1 

4 

2 

0 

This  is  the  last  milestone  assessment  prior  to  launch.  The  number  should  not  change 
between  this  milestone  and  launch.  The  changes  are  not  in  the  measures  of  merit,  but  in 
the  operations  of  those  measures.  For  example,  the  number  of  buildings  will  not  change. 
However,  a  particular  facility  may  lose  power  and  impact  the  operation.  This  will  not 
have  an  effect  on  the  numbers  as  we  have  calculated  them.  That  evaluation  will  be  an 
integral  part  of  the  countdown  process.  The  calculated  Facility  Support  Reliability 
presented  at  the  Launch  Readiness  Review  is: 


Rs(facilities)  =  1  - 


=.73 


This  process  shows  that  a  program  director,  launch  controller,  and  all  other  parties  can 
follow  the  support  reliability  throughout  the  entire  launch  processing  flow.  It  shows  how 
the  reliability  can  change  and  how  it  reacts  to  outside  circumstances.  This  sample  is  a 
fairly  common  approach.  However,  each  program  is  different.  Given  the  proper  software 
tools  a  comparison  to  the  previous  mission  could  be  done  showing  the  significant 
differences  only. 


I 


4..3.3  INTEGRATED  EXAMPLE 

This  same  process  would  apply  to  the  other  parameters  throughout  the  process.  An 
integrated  example  may  look  like  the  following  matrix: 


POSTPRODUCnON 

PRE-VEHCLE-CM 

LAUNCH  SITE 
READINESS 

FLIGHT 
READ  NESS 

LAUNCH 

READINESS 

REVIEW 

STAND  REVIEW 

REVIEW 

REVIEW 

REVIEW 

NUMBER  OF  SUPPORTING 
FACILITIES 


NEW  PERSONNEL  SINCE  LAST 
LAUNCH 


NUMBER  OF  LOSSES  SINCE  LAST 
LAUNCH 


1 

1 

2 

1 

1 

1 

AIR  FORCE 
CREW 


NEW  PERSONNEL  SINCE  LAST 
LAUNCH 

5 

5 

NUMBER  OF  LOSSES  SINCE  LAST 
LAUNCH 

5 

5 

LENGTH  OF  SHIFTS  WORKED 


LENGTH  OF  LAUNCH  CYCLE 


USCEPTIBILITY  OF  MISSION  TO 
ATHER 
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Assuming  the  relationships  described  earlier,  the  Rs(sum)  would  be  the  following  for  each 
review  milestone. 


Rs  Value  Summary 

POCTPRODUCTION 

REVIEW 

PRE-VEHCLErON- 
ST AND  REVIEW 

LAUNCH  SITE 
READINESS 
REVIEW 

FLIGHT 

READINESS 

REVIEW 

LAUNCH 

READINESS 

REVIEW 

FACnmES 

0.27 

0.53 

0.47 

0.67 

0.73 

CONTRACTOR 

CREWS 

0.56 

0.52 

0.48 

0.60 

0.64 

ATRFOBCE 

CREWS 

0.44 

0.36 

0.36 

0.36 

0.56 

RANG£ 

0.51 

0.43 

0.40 

0.43 

0.40 

WEATHER 

0.55 

0.40 

0.35 

0.40 

0.45 

0.47 

0.45 

0.41 

0.49 

0.56 

From  this  example,  we  can  clearly  see  how  the  Rs  value  fluctuates  throughout  the  launch 
processing  cycle.  Although  a  more  thorough  examination  of  the  various  related  factors 
would  show  additional  parameters  to  monitor,  this  example  shows  a  fair  representation  of 
Rs. 


4.4  TRUE  RELIABILITY  CALCULATIONS 

The  example  in  Section  4.3  is  a  hypothetical  example.  The  best  practice  is  to  have  several 
metrics  monitored  on  a  periodic  basis.  Section  4.3  shows  a  method  for  manual  calculation 
of  the  Rs  value.  Ideally,  it  would  be  calculated  using  computers  monitoring  specific 
functions.  For  example,  all  the  crew  data  would  be  in  a  database.  The  Rs  calculation  for 
the  crew  would  draw  its  references  from  the  database. 

Additionally,  a  history  of  events  is  necessary  to  perform  a  quality  Rs  calculation.  Until 
that  history  is  available,  a  simpler  calculation  is  appropriate.  However,  once  a  data  set  of 
significant  size  is  developed,  then  the  reliability  calculations  should  become  more  rigorous 
and  more  typical. 

The  reliability  function  is  typically  determined  from  the  probability  of  system  success.  The 
reliability  function,  R(t),  is  defined  as 


R(t)  =  1  -  F(t) 

with  F(t)  as  the  probability  of  failure  during  time  t.  F(t)  is  the  failure  distribution  function. 
If  the  variable  t  is  described  by  f(t),  then  the  expression  for  reliability  is 


«(0  =  1-F(0  =  //(<)* 


describing  the  time  to  failure  with  a  exponential  density  function  gives 

where  0  is  the  mean  life  and  t  the  period  of  interest.  So,  the  reliability  at  time  t  is 

R(t)  =  ]j*-tl6dt  =  e-tie 

Mean  life  (0)  is  the  arithmetic  average  of  the  lifetimes  of  all  items  considered  which  for 
the  exponential  function  is  Mean  Time  Between  Failure  (MTBF).  So, 

R(t)  =  e~t,M  =  e~* 

where  X  is  the  instantaneous  failure  rate  and  M  the  MTBF. 

This  approach  identifies  a  typical  calculation  for  determining  a  reliability  value  for  a  given 
period  of  time.  Therefore,  the  same  approach  applies  to  the  matrix  in  Section  4.3  for 
space  launch  systems.  Obviously,  for  this  calculation  to  be  useful,  there  needs  to  be  a 
sophisticated  database  of  historical  values.  Determining  the  Mean  Time  Between  Failures 
for  each  of  the  support  parameters  is  a  significant  effort.  However,  it  is  necessary  to 
accurately  assess  the  Rs. 

A  launch  system  is  a  combination  of  series  and  parallel  networks.  Traditional  definitions 
of  these  are: 


Series  Networks 

In  a  series  network  all  components  must  operate  in  a  satisfactory  manner  for  the 
system  to  function.  If  the  system  includes  subsystem  A,  subsystem  B  and 
subsystem  C,  the  reliability  of  the  system  is  the  product  of  the  reliabilities  of  the 
individual  subsystems 


re  liability  (R)  =  (RA)(RB)(RC) 


Parallel  Networks 

A  parallel  network  is  one  where  a  number  of  the  same  components  are  in  parallel 
and  where  all  the  components  must  fail  in  order  to  cause  total  system  failure.  For 
a  system  with  subsystems  A  and  B  the  reliability  is  expressed  as: 

reliability^ )  =  (RA+RB)~  (RA  )(RB  ) 


Further  studies  could  determine  how  the  various  parameters  within  each  support  factor 
defined  in  Section  4. 1  relate  to  each  other.  Some  are  parallel  and  some  are  serial. 


5.0  CONCLUSIONS 

The  space  launch  industry  needs  a  more  complete  reliability  monitoring  system.  This  is 
essential  as  the  launch  rates  decrease.  Air  Force  launch  crews  can  take  simple  to  assist  in 
building  a  true  reliability  capability. 

The  universal  data  set  for  space  launch  systems  is  statistically  insignificant  to  warrant 
exclusion  of  factors  others  may  consider  third  or  fourth  order.  These  factors  include 
calculations  of  the  reliability  of  more  than  simply  the  launch  vehicle  hardware.  Based  on 
studies  previously  performed  by  the  Air  Force,  this  data  set  should  include: 

O  Facilities 
O  Contractor  Crew 
3  Air  Force  Crew 
O  Range 
O  Weather 

There  may  be  other  factors,  but  these  are  critical  for  beginning  to  develop  a  support 
reliability  system.  These  factors  have  specific  parameters,  or  metrics,  that  are  tracked 
against  a  scale.  These  scaled  values  allow  program  managers  to  make  a  complete 
reliability  assessment  throughout  the  launch  processing  cycle.  This  Actual  Reliability  (Ra) 
is  a  function  of  both  the  launch  vehicle  Reliability  (Rv)  and  the  support  system  Reliability 
(Rs).  The  exact  relationship  needs  to  be  determined  for  each  launch  system. 

Ra  should  be  monitored  throughout  the  launch  processing  cycle.  It  should  be  a  vital  part 
of  the  decision  making  process  for  launch.  After  a  few  launches,  program  managers  can 
compare  the  current  launch  to  previous  launches.  This  adds  confidence  in  the  ability  of 
the  entire  system. 


6.0  RECOMMENDATIONS 

1.  The  Air  Force  should  implement  a  reliability  monitoring  and  tracking  system.  This 
system  should  track  key  parameters  within  the  factors  described  in  Section  4. 1 . 

2.  Further  studies  should  be  done  to  determine  the  actual  relationships  between  the 
various  parameters.  This  study  should  define  where  the  serial  and  parallel  relationships 
exist.  It  should  also  identify  the  proper  weighting  of  the  various  parameters.  For 
example,  if  the  facilities  are  twice  as  important  as  the  contractor  crew,  then  it  would  be 
weighted  by  a  factor  of  2.  This  would  define  the  proper  weighting  of  the  various 
parameters. 

3.  One  launch  system  should  be  identified  for  a  test  case  for  implementing  this  reliability 
approach.  This  could  be  done  on  a  non-interference  basis.  As  the  launch  processing  was 
completed,  the  various  reliability  factors  could  be  tallied.  This  would  give  the  Air  Force 
an  indication  of  success  of  this  approach. 


APPENDIX  A 


ACRONYMS 

CAMS . Core  Automated  Maintenance  System 

EMDAS . Enhanced  Missile  Data  Acquisition  System 

MTBF . Mean  Time  Between  Failures 

OJT . On  the  Job  Training 

Ra . Actual  Reliability 

Rs . Support  Reliability 

Rv . Vehicle  Reliability 
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